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ABSTRACT 

A flight test program was conducted in conjunction 
with a ground-based piloted simulation study to enable a 
comparison of handling qualities ratings for a variety of 
maneuvers between flight and simulation of a modem 
high performance airplane. Specific objectives included 
an evaluation of pilot-induced oscillation (PIO) 
tendencies and a determination of maneuver types which 
result in either good or poor ground-to-flight pilot 
handling qualities ratings. 

A General Dynamics F-16XL aircraft was used for the 
flight evaluations, and the NASA Langley Differential 
Maneuvering Simulator was employed for the ground 
based evaluations. Two NASA research pilots evaluated 
both the airplane and simulator characteristics using 
tasks developed in the simulator. Simulator and flight 
tests were all conducted within approximately a one 
month time frame. 

Maneuvers included numerous fine tracking evaluations 
at various angles of attack, load factors and speed 
ranges, gross acquisitions involving longitudinal and 
lateral maneuvering, roll angle captures, and an ILS task 
with a sidestep to landing. Overall results showed 
generally good correlation between ground and flight for 
PIO tendencies and general handling qualities 
comments. Differences in pilot technique used in 
simulator evaluations and effects of airplane 
accelerations and motions are illustrated. 

NOMENCLATURE 

g normalized acceleration 

p roll rate, deg/sec 

p roll rate acceleration, deg/sec 2 

q pitch rate, deg/sec 

r yaw rate, deg/sec 

a angle of attack, deg 


0 

pitch angle, deg 

<t> 

jinnrpvi Qttr 

roll angle, deg 

me 

ftWIvVlrtUV 

AGL 

height above ground, ft 

CDI 

course deviation indicator 

FCS 

flight control system 

GPS 

global positioning system 

HQ 

handling qualities 

HQR 

handling qualities rating 

HUD 

heads-up display 

ILS 

instrument landing system 

KCAS 

calibrated airspeed, kt 

mR 

milliradian 

NM 

nautical mile 

PIO 

pilot-induced oscillation 

TACAN 

tactical air navigation set 


INTRODUCTION 

Flight simulation is increasingly being relied upon in 
the design of new aircraft or modifications to existing 
airplanes due to the high cost of flight test. The 
requirements for effective simulation capabilities and the 
uses of simulation have increased rapidly due to the 
reduction of funds available for prototyping and the 
increased complexity of modem airplane flight 
systems 1 . Historically, piloted simulations have been 
used very effectively in systems studies, identifying 
control system problems in gain scheduling, rate 
limiting, or other control law implementation errors 2 . 
Actual prediction of flight handling qualities (HQ) 
using ground-based simulation studies, however, has 
been less successful. Issues contributing to frequent 
poor correlation between ground-based and flight 
handling qualities include psychological, physical, and 
physiological factors 3 . Many rules of thumb have been 
arrived at, usually discounting simulation HQ results 
because experience has shown that the airplane has 
usually been easier to fly than the simulator. Another 
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area in which simulation has not been reliable is the 
prediction of pilot-induced oscillation (PIO) 
characteristics in flight. 

In order to rely on simulation in the design and 
evaluation of new airplane configurations* the strengths 
and limitations of the tools need to be understood. 

Many challenges are present when using ground based 
simulation to evaluate flight handling qualities. Pilots 
generally are more attentive to workload than task 
performance when using HQ evaluation scales 4 . This 
implies that the perception of the HQ is strongly related 
to visual or motion cues which help or hinder the pilot 
in performing the task - regardless of the actual airplane 
response characteristics 5 . In fact* most measures used to 
predict longitudinal HQ and PIO characteristics involve 
pitch rate* pitch rate acceleration* or load factor 
response 6,7 which may be very difficult to present to a 
pilot in ground-based simulation. The present study 
provides direct correlation of fixed-base ground 
simulation with flight results using the same 2 pilots, 
and the same maneuvers in both flight and simulation 
evaluations. The flight and simulation tests were 
conducted concurrently to obtain the best possible 
correlation between simulation and flight. The data 
obtained included pilot comments and ratings as well as 
time history information of control inputs, aircraft 
states and other parameters obtained from the 
instrumented aircraft. 

DESCRIPTION OF AIRCRAFT 

The airplane utilized in this investigation was a General 
Dynamics F-16XL-1 (Figure 1). This airplane is a 
derivative of the F-16 made by extending the fuselage, 
removing the horizontal tails, and incorporating a 
highly swept cranked delta wing and is one of two F- 
16XL’s built. F-16XL-1 is a single-place airplane, and 
F-16XL-2 was built as a two-place airplane. The test 
airplane was highly instrumented for aerodynamic and 
stability and control flight tests, and included air data, 
global positioning system (GPS) data, Euler angles, 
angular rates, linear accelerations, control surface 
positions, and control input force measurements 
recorded in flight and telemetered to the ground station. 
Control effectors are elevons and ailerons at the wing 
trailing edge, and a rudder. Leading edge flaps are also 
used which are scheduled with angle of attack and Mach 
number. The flight control system is very similar to 
the F-16A, with a limited displacement force stick 
controller. The flight control system imposes an angle 
of attack limit of approximately 30° in normal mode, or 
20° in a special reduced angle-of-attack mode which was 


originally implemented for specific flight test envelope 
expansion reasons. 

The longitudinal flight control laws provide a g- 
command system with the landing gear up and at of s 
below approximately 19° in the normal mode, and 
below a = 10° in the reduced a mode. For a’s > 19°, 
(or 10° in reduced a mode), the pitch command is a 
blend of normal acceleration and angle of attack. With 
the landing gear down, the flight control laws transition 
to a pitch rate command system which blends angle of 
attack in above a = 10°. The lateral flight control laws 
provide a roll rate command system. The roll rate 
command limit is reduced at airspeeds below 250 
KCAS, a < 15°, normal acceleration > 6 g’s, or with 
the landing gear extended. The flight control system of 
the airplane is summarized in reference 8. 

DESCRIPTION OF SIMULATOR 

The simulation studies were conducted in the NASA- 
Langley Differential Maneuvering Simulator, which is a 
fixed base 40-foot diameter dome cockpit simulator 9 . 
Inflatable seat cushions were employed for g-cueing, 
however no other motion cueing devices were used. 

The cockpit was fitted with an actual F-16 sidestick 
controller. The primary flight instruments were the 
heads-up display (HUD) which was configured to 
represent the F-16XL display, and an angle of attack 
tape, similar to that in the airplane. The simulation 
math model was originally developed by General 
Dynamics from wind tunnel data and was subsequently 
updated from flight test results. Before these simulation 
tests were flown, a validation study of the simulator 
was undertaken and time history responses of the 
simulation were judged to be very representative of the 
airplane. The simulation math model was a full 
envelope representation of the airplane including angle 
of attack, sideslip, Mach and altitude effects on the 
aerodynamic characteristics. 

The simulation used a computer generated visual 
imagery system for nearly 360° field of view. The 
target aircraft used for the tracking and gross acquisition 
evaluation tasks was displayed on the dome by a 
separate projector system. 

TEST TECHNIQUES 

Evaluation test maneuvers were selected using the 
simulator. Tasks were selected which involved high 
gain piloting tasks in order to evaluate the flight 
characteristics in the longitudinal and lateral axis over a 
wide range of flight conditions. Additionally, PIO 
characteristics were evaluated by specifying close 
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tolerances required for the tracking and capture tasks 
selected. Pilot comments were obtained in flight and at 
the post flight debrief following a question guidelines 
card which was developed during initial simulator 
evaluations. In addition to general comments on items 
such as adequacy of rates, initial response 
characteristics, predictability of response, undesirable 
motions/coupling, etc., the pilots gave handling quality 
ratings (HQR) based on the Cooper-Harper scale 11 in 
accordance with the criteria established for each of the 
maneuvers. Additionally, each of the maneuvers was 
assessed with a PIO scale rating from the scale shown 
in table 1 . 

Several maneuvers were selected from the STEMS 
maneuver set 11 and modified to be applicable in the 
lower angle of attack range at which these tests were 
conducted (below 30°). All of the target tasks and 
initial setups were developed in the simulator and then 
were used in both the simulation and flight evaluations. 
The primary flight instrument used in the evaluations 
was the HUD. For tracking and gross acquisition tasks, 
the HUD displayed a reticle which was depressed 3° 
below the aircraft reference line. The reticle had inner 
and outer rings of 50 mR and 100 mR diameters, 
respectively. The target aircraft which was available for 
the flight evaluations was a Northrop T-38A. The 
target T-38A was equipped with a GPS receiver to allow 
post-flight analysis of the relative positions of the two 
airplanes. Because of antenna placement on both the F- 
16XL and T-38A, GPS data was unavailable for most of 
the maneuvering flight conditions which involved 
substantial bank angles. Air-to-air TACAN and visual 
references were used to setup initial conditions for the 
evaluation tasks in flight. Because of the dissimilar 
performance capabilities of the two aircraft, some of the 
tracking tasks involved special setups which were 
developed in the simulator and subsequently worked 
well in flight. A list of the maneuvers evaluated and 
their associated rating criteria is shown in table 2. 
Selected data from these tests will be shown. 

Two NASA research pilots conducted the flight and 
simulator evaluations. Flight and simulation tests were 
conducted concurrently to provide the most valid 
comparisons possible between simulation and flight. 

RESULTS AND DISCUSSION 

Correlation of handling qualities ratings, pilot 
comments and analysis of flight and simulated time 
histories of selected maneuvers will be presented in the 
paper for the following tasks: gross acquisition, steady 
tracking, roll angle captures, and the ILS landing 
approach. Where significant differences existed between 
pilots, data will be shown for both pilots; otherwise, 


representative data from one pilot will be presented. All 
of the maneuvers (except the ILS) were initiated at an 
altitude of 15,000 ft. Table 3 lists a summary of pilot 
ratings for each maneuver discussed. 

GROSS ACQUISITION 

Two types of gross acquisition tasks were evaluated. A 
purely longitudinal acquisition and a multi-axis 
acquisition. These maneuvers were set up at various 
speeds and separation distances to evaluate the airplane 
characteristics across a range of g-loading, pitch rate, 
and acquisition geometry dynamics. Specific 
longitudinal gross acquisition examples will be 
discussed using maneuvers 2.1 and 2.4 from table 2. 

The maneuver required 3 acquisitions of the target, each 
followed by reducing 0 to the horizon. Maneuver 2.1 
was initiated at approximately 150 KCAS and a = 15°. 
The target was co-speed and ahead and above by 6000 
and 5000 ft., respectively. The maneuver resulted in a 
large amount of time on the angle-of-attack limiter of 
30° during the capture. Maneuver 2.4 was initiated at 
an airspeed of approximately 240 KCAS and a = 7.5°. 
The target was ahead and above by 6000 and 2000 ft., 
respectively. The maneuver maximum angle-of-attack 
was approximately 20°. 

Pilot HQR ratings for maneuver 2.1 show a discrepancy 
between the two pilots. Pilot A evaluated the 
simulation better than flight, whereas pilot B evaluated 
the simulator worse. Pilot A comments were that in 
flight there was large amplitude PIO, particularly on the 
second pull to the target. In the simulator, comments 
were that after some learning, he could arrive on target 
with only one overshoot, but that the target had a 
tendency to “bounce” out of the reticle. Figure 2 shows 
a comparison of the maneuver between simulation and 
flight. Flight data is shown for 2 captures, and 
simulation data is shown for 1 capture. Figure 2 shows 
the angle-off the nose during the gross acquisition for 
pilot A. The data show an initial overshoot of the 
target airplane during the captures in both simulation 
and flight, however the flight results show more 
oscillations while trying to keep the target in the 
reticle. The second capture (flight) shows a divergent 
PIO with overshoots and undershoots getting 
progressively larger. The control input time histories, 
(figure 2), give some insight into the differences 
between the simulation and flight maneuvers. As pilot 
comments indicated, in the simulator, the pilot was able 
to “learn” the task after several practice attempts and 
developed a procedure to accomplish it. This involved a 
large (full-scale) initial input then a release of the stick 
at an appropriate lead angle. After some time to see 


3 



where the airplane settled out, the pilot again began 
making corrective inputs to keep the target in the 
desired location. In flight, the pilot did not have the 
opportunity to repeatably fly the exact same task over 
and over to develop a “canned” technique. As a result, 
the pilot was constantly in the control loop with 
control forces in both directions resulting in oscillations 
relative to the target. In the second capture, the pilot 
entered a divergent PIO using full aft control forces. 

Pilot B rated the simulator worse than flight, and also 
rated the flight HQ much better than pilot A. It should 
be noted that pilot B conducted only one capture for this 
maneuver in flight, compared to the two conducted by 
pilot A. Figure 3 shows the comparison of flight and 
simulator performance of pilot B for maneuver 2.1. 

The angle-off nose data shows that both simulation and 
flight exhibited oscillations in the capture dynamics 
resulting in slight excursions out of the desired capture 
criteria. Pilot comments were very similar for both 
flight and simulation tasks. During evaluations in the 
simulator and in flight, pilot B commented that the 
pitch rate was slow and that the configuration had PIO 
tendencies. 

One tendency seen in common to both pilots between 
simulation and flight was that in the simulation, both 
pilots had a tendency to use larger pitch inputs early in 
the maneuvers. This resulted in significantly larger 
maximum pitch rates in the simulation as shown in 
figure 4. Pilot A used 80% higher maximum pitch rate 
for the first capture in the simulator compared to flight, 
while pilot B used 225% higher maximum pitch rate. 
The much larger pitch rates used in the simulation by 
pilot B certainly could be a factor for the decreased HQ 
ratings in the simulation relative to flight. Absence of 
tactile cues are probably a primary cause for the 
difference in pilot technique between simulation and 
flight for this maneuver. At these low speeds, the pitch 
rate is low, providing poor visual feedback in the 
simulator, which combined with the lack of g or pitch 
rate acceleration cues result in poor correlation between 
simulation and flight. 

At higher speeds, the agreement both between pilots and 
between simulation and flight improved. Figure 5 
shows a comparison of maneuver 2.4 in the simulator 
and in flight for pilot A. Both simulation and flight 
show very oscillatory behavior during the capture. 
Control forces for both simulation and flight have a 
similar frequency and are indicative of a high pilot 
workload. Pilot comments for flight were that there 
was usually one overshoot (time history data show 
more) and that it was PIO prone. In the simulator, the 
comments were that the response was oscillatory and 


that it was similar to the airplane. A constant 
amplitude PIO developed when attempting to do 
aggressive captures. If the maneuver was performed less 
aggressively, then the target could be captured with one 
or two overshoots. Successful captures required 
substantial lead in taking out the control input. 

Figure 6 shows a comparison of pitch rates used during 
maneuver 2.4 in the simulator and in flight for both 
pilots. Again, both pilots used considerably more 
initial pitch rate in the simulation than in flight. For 
this maneuver, both pilots had excess control command 
available, and the captures were occurring at 
considerably lower a’s (about 15° vs. 25°) than for 
maneuver 2. 1 . The improved agreement both between 
pilots and between flight and simulation for maneuver 
2.4 compared to maneuver 2. 1 could be related to the 
increased sensible cues presented to the pilot. At the 
higher airspeed of maneuver 2.4, pitch rate, acceleration, 
and load factor all increased. This allowed more tactile 
feedback in flight. In the simulator, the increased pitch 
rate allowed better control through increased visual 
feedback due to the higher pitch rates, which resulted in 
better relationships between flight and simulator pilot 
inputs. Additionally, neither pilot saturated the stick 
command in flight or in the simulator. 

Multi-axis gross acquisitions were conducted to include 
effects of lateral maneuvering. The maneuver placed the 
target abeam and flying away from the F-16XL at a 90° 
heading difference at 160 KCAS. In general, lateral- 
directional HQR’s for both flight and simulation were 
very similar. Longitudinal ratings were not as 
consistent between flight and simulation as seen in the 
previous maneuver task. Relatively large variability 
between the pilots for a given maneuver was also seen. 
Figure 7 compares lateral stick force and roll angle time 
histories between simulation and flight for maneuver 
6.2 for pilot A. As can be seen, very similar roll 
angles and rates, and control activity were achieved for 
both flight and simulation. A similar comparison was 
made with the longitudinal stick force and load factor in 
Figure 8. As can be seen, the simulator was flown with 
considerably more stick input magnitude resulting in 
higher load factors. Focusing on the first capture (at 
time = 18-28 sec), the lateral control inputs and 
associated roll rates were similar between simulation 
and flight. The longitudinal characteristics of the data 
were very different during the capture as shown in figure 
9. The simulator task resulted in large amplitude 
oscillatory control inputs and load factor response. 

Flight results show smaller amplitude control inputs 
and associated load factor responses. These differences 
may be attributable to the g-cues in flight allowing for 
smoother aircraft control. Pilot comments for the 
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simulation indicated that it was difficult to separate 
longitudinal and lateral problems, but that there were 
several overshoots, and very bad PIO (although no PIO 
rating was assigned). Flight comments were very 
similar, however the pilot noted that overshoots were 
primarily side-to-side overshoots in the pull-up and roll 
maneuver and chose to blame the lateral axis primarily 
for the overshoots. 

Pilot B flew the lateral portion of maneuver 6.2 very 
similarly between simulation and flight, as did pilot A. 
Additionally, pilot B flew the longitudinal part of the 
task very similarly between the simulation and flight as 
shown in figure 10. This resulted in very similar HQR 
ratings between simulation and flight. Pilot comments 
indicated that pilot B downgraded the flight longitudinal 
HQ rating from 4 to 5 based on his perception of 
undesired slow pitch rate resulting in excessive time to 
capture the target. 


TRACKING 

Fine tracking tasks were conducted to evaluate the 
characteristics of the airplane at various speeds, angle of 
attack and g-loading conditions. Rating criteria were 
selected to excite PIO tendencies if any existed. 

Tracking tasks included tracking steady targets and 
reversing targets. Each of the maneuvers were flown 
with a separation distance of approximately 1000 ft. 

During the steady tracking evaluations, both pilots 
consistently rated the lateral handling qualities of both 
flight and simulation as meeting desired criteria (HQR 
ratings 3 or 4). Both pilots also rated the airplane as 
having longitudinal PIO tendencies, with PIO scale 
ratings of 2 or 3 across the speed and g range tested. In 
the simulator, pilot A also rated PIO characteristics the 
same as flight, however pilot B rated all conditions at 
350 KCAS or above as having no PIO tendencies (PIO 
scale = 1). A detailed look at the pilot comments and 
data however, reveal that oscillations were occurring in 
all conditions, but at the higher speeds, pilot B did not 
identify the oscillations as PIO. 

Typical data for steady tracking at 200 KCAS 
(maneuver 3.4) is shown in figures 1 1 and 12. Data for 
pilot A shows continuous oscillations in load factor and 
stick force in figure 1 1 . The data show nearly identical 
magnitudes and frequency of both load factor and stick 
input for both simulation and flight. Figure 12 shows 
the results for pilot B for the same maneuver. This data 
reveal much higher frequency and amplitude results of 
longitudinal stick inputs and load factor response during 
flight compared to simulation. The flight 


characteristics obtained by pilot B look very similar to 
the simulation and flight results from pilot A, however 
due to learning, reduced stimulus or some other factor, 
pilot B approached the task differently in the simulator. 

Tracking a target during reversals was also evaluated. In 
the simulator, the target reversed course after a heading 
change of 30° for KCAS < 1 80, and after 60° at higher 
speeds. In flight, the evaluation pilot called for 
reversals. General results were that both simulation and 
flight produced PIO tendencies at the higher speeds. As 
speed decreased, longitudinal handling qualities 
improved for tracking. This is evident in the PIO scale 
ratings as shown in figure 13. Results obtained in this 
test show that the simulator tended to slightly over- 
predict PIO tendencies, especially at low speeds. 

Another trend from this data is that PIO tendencies 
usually resulted in worse HQR’s at high speeds in the 
simulator than in flight. Pilot comments for maneuver 
5.1 (150 KCAS) in the simulator were that moderate 
inputs tended to excite pitch PIO, but the PIO could be 
suppressed with increased pilot workload. Though 
workload was elevated to avoid oscillations, desired 
performance could be achieved. In flight, comments 
indicated occasional PIO or “bobbles” following 
corrective inputs, but otherwise solid desired 
performance. At higher speeds, maneuver 5.5, 
simulator HQR’s were significantly worse than flight 
ratings, however the pilot comments were very similar. 
Both pilots’ comments in the simulator included 
continuous PIO, sawing through the CG of the target, 
and target bouncing out of reticle. In flight comments 
were that there was continual moderate PIO. 

Comparison of data between simulator and flight for 
maneuver 5.5 (figure 14) show increased amplitude of 
longitudinal inputs in the simulator, but the inputs 
were at a similar frequency as observed in flight. 
Although angle-off data is not available from flight due 
to GPS limitations previously discussed, simulator 
results in figure 15 show continuous oscillations during 
the tracking, but even while reversing course, the target 
remained within the 50 mR reticle. 

ROLL ANGLE CAPTURES 

<(> CAPTURES 1-g: 

Aircraft bank angle (<(>) captures were conducted to 
evaluate lateral handling qualities and control of the 
aircraft at four different angles of attack, which were 
produced by varying maneuver speed and are listed in 
table 2. The maneuver was initiated while wings were 
level and at constant speed and altitude. Bank angles 
were captured in a sequence of -60 (left bank), 60 (right 
bank), and 0 degrees. The three sub-phases of the 
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maneuver are referred to as maneuver entry, roll back, 
and roll out. Maneuver 1 .4 could not be accomplished 
at constant airspeed due to large amounts of nose down 
control being generated by the control system as a 
response to roll rate and was only attempted by one of 
the pilots in flight. 

Analysis of the resulting data for both simulated and 
actual flight showed desired performance for all bank 
angle captures attempted. Additionally, pilot inputs, 
and aircraft performance were similar for both simulated 
and actual flight results. Figure 16 shows bank angle, 
roll rate, roll rate acceleration, and pilots lateral stick 
input for maneuver 1.2 and is representative of all other 
1-g <)> captures for both pilots. From this figure the 
similarities between flight and simulator results are 
obvious. An evaluation of maximum roll rate and 
maximum roll rate acceleration for the three sub-phases 
of the maneuver was performed and results for flight are 
plotted against results obtained for an identical 
maneuver sub-phase from simulation in figure 17. 

Lines representing values which are equal to, ± 20% and 
± 40% of simulator results are included in figure 17 as a 
guide to interpret the comparison. From this figure it 
can be seen that a majority of points fell below the line 
where flight results were equal to simulator results for 
maximum roll rate and roll rate acceleration. Table 4 
presents a distribution of maximum roll rate and roll 
rate acceleration values in flight relative to simulation 
maximum values. 

From this table it can be seen that generally pilots used 
less roll rate and roll rate acceleration in flight than in 
simulation. This result can be attributed to the lack of 
motion cues in the fixed-base simulation which are, of 
course, present in flight. However, motion cues are 
usually assumed to affect roll rate acceleration, which 
the pilot feels, not roll rate which the pilot sees. The 
above table and associated figure show much more 
significant reductions for flight results in roll rate than 
roll rate acceleration, which is an unexpected result. 

Pilot remarks and handling quality ratings were similar 
for both simulator and flight. However, there was some 
perception by the pilots that adequate performance was 
not being achieved especially in the simulator. As 
mentioned above, desired performance was achieved for 
all bank angle captures both in flight and in the 
simulator. Pilot remarks indicated that lateral sideforce 
cues combined with outside visual cues made it much 
easier to evaluate roll characteristics such as roll rate and 
damping, and to feel roll rate non-linearities. Pilots 
also commented that technique was important and that 
the limited displacement force stick made it difficult to 
determine maximum control inputs. For the two 


slower airspeeds (KCAS = 100 and 130), pilots 
downgraded the maneuver by increasing the resulting 
HQR 2 to 3 units as a result of less than acceptable roll 
rate available. This level of HQR degradation was 
common for both simulation and flight. The HQR 
values presented in table 3 were results obtained without 
consideration of the roll rate available criteria. 

0 CAPTURES elevated-g: 

Elevated-g 0-captures were conducted to evaluate lateral 
handling qualities and control in conjunction with a 
longitudinal task of maintaining constant normal g- 
loading. Different combinations of aircraft speed and g- 
loading were employed to evaluate aircraft handling 
qualities for an angle of attack range from 7 up to 18 
degrees, and are listed in table 2. The maneuver was 
initiated from left-wing-down turning flight at constant 
speed and slightly varying altitude. Pilots adjusted 
longitudinal stick force to obtain the specified initial g- 
loading at 0=-6O (left bank) degrees. Bank angles were 
captured in a sequence of 60 (right bank), and -60 
degrees. The two sub-phases of the maneuver are 
referred to as maneuver roll back, and roll out. 

Analysis of the resulting data for both simulated and 
actual flight showed desired lateral performance for all 
bank angle captures attempted. Additionally, pilot 
inputs, and aircraft performance were similar for both 
simulated and actual flight results. Figure 18 shows 
data from maneuver 8.3 and is representative of all of 
the elevated-g 0 captures flown. From figure 1 8a, the 
similarities between flight and simulator results are 
obvious for lateral performance. Similarities also exist 
for longitudinal performance (figure 18b), however the 
pilots did not obtain desired accuracy for most of the 
maneuver, and even experienced some short excursions 
beyond adequate limits. The relatively poor 
longitudinal performance was partially due to the 
inadequacy of the HUD display for this task. 

An evaluation of maximum roll rate and maximum roll 
rate acceleration for the two sub-phases of the maneuver 
was performed. Figure 19 presents results for flight 
plotted against results obtained for an identical 
maneuver sub-phase from simulation. Lines 
representing values which are equal to, ± 20%, and ± 
40% of simulator results are included in figure 19 as a 
guide to interpret the comparison. From this figure it 
can be seen that a majority of points fell below the line 
where flight results were equal to simulator results for 
maximum roll rate and roll rate acceleration. Table 5 
summarizes the percentage of points in each plot 
region. 
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From table 5 the same trends seen in the 1-g ^-captures 
are evident - that generally pilots used less roll rate and 
roll rate acceleration in flight than in simulation. A 
total of 87% of the maneuver sub-phases performed in 
flight employed less roll rate than the identical 
maneuver sub-phase performed in simulation as 
compared to 63% for roll rate acceleration. 

Pilot remarks and handling quality ratings were similar 
for both flight and ground based simulation. As 
mentioned above, desired lateral performance was 
achieved for all bank angle captures both in flight and in 
simulation. Pilots remarked that concentrating on 
maintaining g-forces, through the use of the HUD g 
presentation, or a heads-down angle of attack gauge, 
heavily detracted from the bank angle capture 
performance. Relying on body forces to gauge normal 
acceleration also did not yield adequate results for 
maintaining desired or even adequate performance levels. 
Roll rate non-linearities caused by angle of sideslip 
build-up during the maneuver were observed in ground 
based simulation and flight. Lateral sideforce cues 
combined with outside visual cues made it much easier 
to evaluate roll characteristics such as roll rate and 
damping, and to feel roll rate non-linearities. Pilots 
also commented that the simulator “felt” like the 
airplane without the neck-snapping acceleration cues. 

As a result they were less likely to use sudden stick 
inputs in flight, which is also confirmed through 
analysis of the available data. The HQR values 
presented in table 3 were results obtained based on 
lateral performance alone. 

From table 3 it can be seen that pilot B generally rated 
flight one HQR unit better than the same maneuver 
performed in ground-based simulation. Otherwise 
identical HQR results were obtained for all elevated-g <|> 
captures. 

ILS Sidestep Approach to Landing 

An ILS task was evaluated to determine the correlation 
value of a maneuver which involved tight tolerances on 
aircraft flight path control and a large lateral 
repositioning of the aircraft in close proximity to the 
ground. There were three sub-phases of the ILS 
maneuver, which are presented in figure 20. As can be 
seen in this figure a simulated localizer/glideslope 
approach was established with an offset of 840 feet from 
the runway centerline. A diagram presenting important 
elements of the HUD ILS guidance used by the pilots is 
shown in figure 21 . 

The maneuver began in flight with the pilot flying the 
aircraft to a waypoint using the inertial navigation 


system of the aircraft which placed the aircraft a distance 
of two nautical miles, as measured along the intercept 
course, from the localizer intercept point. When this 
maneuver was performed using the ground-based 
simulator, initialization of the maneuver was at the 
waypoint. Once at the waypoint, the aircraft was flown 

on a 30° lateral intercept of the localizer course. At 
approximately 8 NM from the runway threshold an 
intercept of the localizer was performed with the aircraft 
at 2,000 ft. AGL. Intercept of the glideslope occurred at 
about 6 NM from runway threshold. This was the first 
sub-phase of the maneuver and is referred to as 
localizer/glideslope intercept. The second sub-phase 
required the pilot to track the localizer/glideslope course. 
The final sub-phase of the maneuver was the sidestep to 
landing, which was initiated at an altitude of 300 ft. 
AGL. The entire approach was flown without the use 
of a flight director, which increased pilot workload 
compared to normal ILS approaches in this airplane. 

Analysis of the resulting data showed similar 
performance of the maneuver from both flight and 
ground-based simulation. Pilots had little difficulty 
with the localizer/glideslope intercept sub-phase of the 
ILS maneuver with regards to aircraft position. 

However, less than adequate performance of airspeed 
management was noticed for all ILS intercepts 
performed. Pilots tended to focus attention on accurate 
management of trajectory performance rather than on 
precise airspeed control. This could be due to the low 
number of ILS maneuvers performed and minimal speed 
cues afforded to the pilot by the FCS and sidestick 
controller. Desired performance (other than airspeed) was 
achieved during the intercept with only minor 
excursions, which were well within the desired range, 
before stabilization on the localizer/glideslope course in 
flight. In addition, similar bank and pitch angles and 
stick force activity were apparent between simulation 
and flight. 

Desired trajectory performance was accomplished during 
the localizer/glideslope tracking portion of the ILS 
maneuver. Results were very similar between flight and 
simulation, except the speed used in the simulator 
tended to be up to 20 kts. less than used in flight. 

The final sub-phase of the ILS maneuver was the 
sidestep to landing, which was specified to begin at an 
altitude of 300 feet AGL. Figure 22 presents aircraft 
distance from runway centerline, bank angle, and lateral 
stick force as a function of distance from runway 
threshold. Figure 23 presents aircraft altitude, angle of 
attack, pitch attitude, and longitudinal stick force. The 
specified touchdown point was located approximately 
1,049 feet down the runway and on the centerline. As 
can be seen in figure 22 and 23, pilot workload 
increased dramatically as pilot progressed through the 
sidestep maneuver. Initially pilot activity was mainly 
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an increase in lateral commands. However, as the pilots 
began rolling out of the second part of the sidestep 
maneuver, pitch activity increased rapidly for flight and 
was not as pronounced in the ground-based simulation 
data as can be seen in figure 23. Pilots tended to not be 
as aggressive to acquire the designated touchdown point 
in ground-based simulation as can be seen by the more 
gradual flare in the altitude results. Neither ground- 
based simulation or flight results produced desired, or 
even adequate results for touchdown distances. Flight 
results were much closer to the desired touchdown point 
than for ground-based simulation with and average of 
540 ft. short of the specified touchdown point for flight 
as compared with 2,203 ft. passed the desired touchdown 
point for ground-based simulation. Although very few 
ILS task evaluations were conducted, they do reveal 
trends. Pilots were capable of controlling touchdown 
rate better in flight, in spite of landing before the 
specified touchdown point, and averaged approximately 
2.5 ft/sec as compared with 3.4 ft/sec for simulation. 

Pilot ratings for the localizer/glideslope intercept and 
tracking sub-phases of the maneuver were almost 
identical in flight and simulation. Comments indicated 
the maneuver was not difficult to perform and generally 
attained desired performance both in flight and in 
ground-based simulation. HQR ratings of 3 were 
generally assigned by both pilots for the two initial 
sub-phases of the ILS maneuver for lateral and 
longitudinal tasks. Pilot A rated the lateral part of 
localizer/glideslope tracking a 4 due to continual S- 
tuming across the desired course. The sidestep sub- 
phase of the ILS maneuver received a HQR of 7 by both 
pilots for ground-based simulation and from pilot A in 
flight. Pilot B rated the sidestep maneuver in flight a 
HQR of 4, but began the maneuver higher than the 
specified 300 ft. AGL. Comments cited the need to go 
below the previously tracked glideslope flight path to 
touchdown at the specified point as a significant 
problem. The touchdown point was specified to be 
adjacent to where the glideslope intersected the ground. 
Pilot B commented that the sidestep maneuver should 
have been started at 400 to 500 AGL instead of 300 
feet. This pilot aborted his first flight attempt of the 
sidestep maneuver, which was initiated at 300 ft. AGL, 
at an altitude of approximately 40 ft. AGL. Both pilots 
commented that lack of clear peripheral visual cues, 
inherent to the dome simulator visual system, impaired 
the ability to judge the height above the runway and 
made precision landing impossible. No comments were 
made regarding the lack of motion cues for ground-based 
simulation as a limiting factor. 


CONCLUDING REMARKS 

A flight test validation of several handling qualities 
tasks has been conducted on a high performance airplane 
with a highly swept wing to compare with results from 
a fixed-base dome simulator. These results give 
guidance to which classes of evaluation maneuvers are 
likely to give comparable results in the simulator, and 
what differences one may expect when evaluating an 
airplane using piloted simulation prior to flight, or in 
the design stage. Specific results are as follows: 

1 . Task learning in the simulator can result in better 
handling qualities ratings in simulator than in flight. 

2. Pilot input magnitudes, especially in pitch, tend to 
be much greater in the simulator than in flight - 
particularly at lower airspeeds - probably due to reduced 
feedback cues. At slow speeds, pitch rate is slow, so 
visual feedback in the simulator is weak. 

3. Lateral inputs are similar across the speed range. 

The relatively high roll rates provide good visual cues 
in the simulator and show that for lateral control, visual 
cues are probably more important than acceleration 
cues. 

4. When the simulation is flown with initial inputs 
similar to flight, much better agreement between flight 
and simulation HQR’s result. This indicates that pilot 
approach can have a large impact on simulator-to-flight 
correlation. 

5. Pilot induced oscillations were observed both in the 
simulator and in flight, and were more pronounced in 
the simulator at slow speeds during the fine tracking 
tasks. 

6. The ILS sidestep maneuver was valuable for 
evaluating a high gain task. Good correlation between 
flight and simulation is evidence that the motion cues 
are not a primary factor in assessing handling qualities 
for this task. The touchdown task was not suitable to 
be used in the simulation for flight correlation. 

Future plans include a more thorough review and 
analysis of the flight and simulation data. A piloted 
simulation study with motion should be conducted to 
evaluate the postulated importance of pitch rate 
acceleration and/or g-cues on low-speed gross 
acquisition tasks. Finally, although this study included 
too few pilot samples to give definitive answers, it has 
shown that simulation-to-flight comparisons can be 
very good, but are highly dependent on pilot technique 
and therefore may vary from pilot to pilot. 
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Description 

Numerical 

Rating 

No tendency for pilot to induce 
undesirable motions. 

1 

Undesirable motions tend to occur 
when pilot initiates abrupt maneuvers 
or attempts tight control. These 
motions can be prevented or eliminated 
by pilot technique. 

2 

Undesirable motions easily induced 
when pilot initiates abrupt maneuvers 
or attempts tight control. These 
motions can be prevented or eliminated 
but only at sacrifice to task 
performance or through considerable 
pilot attention and effort. 

3 

Oscillations tend to develop when pilot 
initiates abrupt maneuvers or attempts 
tight control. Pilot must reduce gain 
or abandon task to recover. 

4 

Divergent oscillations tend to develop 
when pilot initiates abrupt maneuvers 
or attempts tight control. Pilot must 
open loop by releasing or freezing the 
stick. 

5 

Disturbance or normal pilot control 
may cause divergent oscillations. 
Pilot must open control loop by 
releasing or freezing the stick. 
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Table 1. PIO rating scale 
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MANEUVER 

1 Conditions 

DESIRED CRITERIA 

ADEQUATE CRITERIA 

1-g $ - Captures 

# 

KCAS 

a 

2 sec steady at ea. bank 

2 sec steady at ea. bank 


1.1 

200 

To° 

angle (*5°), desired time for 

angle, two overshoots, 


1.2 

150 

15° 

maneuver, one overshoot 

adequate time 


1.3 

130 

18° 




1.4 

100 

25° 



Longitudinal gross 

# 

KCAS 

A0 

Aggressively acquire aim 

Aggressively acquire aim 

acquisition 

2.1 

150 

40° 

point within 50 mR with 1 

point within 50 mR with 2 


2.2 

200 

40° 

overshoot and within 

overshoots and within 


2.3 

240 

40° 

desired time 

adequate time 


2.4 

240 

18° 



Steady tracking 

ft 

KCAS 

a g 

No PIO. Target within ± 5 mR 

Pipper within ± 5 mR 10% 


3.1 

350 

7.5° 2 

of pipper 50% of task and 

of task and within ±25 mR 


3.2 

350 

10° 3 

within ±25 mR remainder of 

remainder of task 


3.3 

470 

10° 5 

task 



3.4 

200 

15° 2 




3.5 

150 

20° 1.5 



Reversals 

ft 

KCAS 

a 

Target within ± 5 mR dia. 

Target within ± 5 mR dia. 

tracking 

5.1 

150 

16° 

pipper 50% of time, ± 25 mR 

pipper 10% of time, ± 25 


5.2 

180 

13° 

remainder of task 

mR remainder of task 


5.3 

200 

15° 




5.4 

240 

12° 




5.5 

300 

9° 



Multi-axis gross 

# 

KCAS 


Target within 50 mR dia. of 

Target within 50 mR dia. of 

acquisition 

6.1 

200 

30° 

pipper with no overshoot 

pipper with 1 overshoot 


6.2 

240 

30° 

and within a desirable time 

and within an adequate 


6.3 

300 

20° 


time 


6.4 

240 

20° 



ILS intercept 


ft 


Altitude within ± 50 feet, 

Altitude ± 150 feet and 





before glideslope intercept, 

speed ± 10 kts. with two 



7a 


and speed to within ±5 kts. 

overshoots greater than 





with no overshoots of one 

one CDI dot or more 





dot or more. 


ILS tracking 


ft 


Maintain localizer and 

Maintain localizer and 





glideslope CDI’s to within 

glideslope CDI’s to within 2 



7b 


one dot, and airspeed to 

dots and airspeed to ±10 kts 





within ± 5 kts 


ILS - sidestep to 


ft 


Touchdown sink rale less 

Touchdown sink rate less 

landing 




than 2.5 feet per second at 

than 5 feet per second at an 



7c 


an angle of attack of ± 1° of 

angle of attack of ± 2° of the 





the target value. Touchdown 

target value. Touchdown 





within ± 20 feet laterally and 

within ± 50 feet laterally an< 





± 60 feet longitudinally of 

± 100 feet longitudinally of 





specified touchdown point. 

specified touchdown point. 

Loaded $ - 

ft 

KCAS 

a g 

2 sec steady at ea. bank 

2 sec steady at ea. bank 

Captures 

8.1 

300 

10° 2 

angle (~5°), desired time for 

angle, two overshoots, 


8.2 

250 

12° 2 

maneuver, one overshoot, 

*4° a (± 0.4 g), adequate time 


8.3 

200 

16° 2 

2° a (± 0.2 g) 



8.4 

200 

13° 1.5 




8.5 

350 

7° 2 




Table 2. Rating criteria 


10 








Task 

Pilot 

Sim long HQR Sim lat HQR 

Sim long PIO Fit long HQR Fit lat HQR 

Fit long PIO 

1.1 

A 

- 

4 

- 

- 

6 

_ 


B 

- 

4 

- 

- 

3 

- 

1.2 

A 

- 

5 

- 

- 

4 

- 


B 

- 

5 

- 

* 

4 

- 

13 

A 

- 

4 

- 

- 

3 

- 


B 

- 

6 

- 

- 

5 

- 

2.1 

_ A 

4 

- 

- 

7 

- 

4 


B 

6 

- 

1 

4 

- 

2 

2.4 

A 

4 

- 

2 

4 

- 

3 


B 

4 

- 

1 

4 

- 

2 

3.4 

A 

6 

3 

3 

4 

3 

3 


B 

5 

4 

2 

7 

3 

3 

5.1 

A 

4 

3 

3 

3 

3 

1 


B 

4 

4 

2 

3 

3 

1 

53 

A 

7 

4 

4 

4 

4 

3 


B 

6 

3 

3 

5 

3 

3 

6.2 

A 

8 

7 

- 

4 

7 

2 


B 

5 

3 

1 

5 

3 

2 

7a 

A 

3 

3 

- 

3 

3 

- 


B 

3 

3 

- 

3 

3 

- 

7b 

A 

3 

3 

- 

3 

4 

- 


B 

3 

3 

- 

3 

3 

- 

7c 

A 

7 

- 

- 

7 

- 

- 


B 

7 

- 

- 

4 

- 

- 

8.1 

A 

- 

4 

- 

- 

4 

- 


B 

- 

4 

- 

- 

3 

- 

83 

A 

- 

4 

- 

- 

4 

- 


B 

- 

4 

- 

- 

3 

- 

83 

A 

- 

4 

- 

- 

4 

- 


B 

- 

4 

- 

- 

4 

- 

8 A 

A 

- 

4 

- 

- 

4 

- 


B 

- 

4 

- 

- 

3 

- 

8.5 

A 

- 

4 

- 

- 

4 

- 


B 

- 

3 

- 

- 

3 

- 

TABLE : 

Handling qualities 






ratings summary 



Flight maximum 


Roll rate 

Roll rate 





values (% of sim 



acceleration 

Flight maximum 1 Roll rate Roll rate 

maximum values 





values (% of sim I acceleration >140% I 7% 0% 


maximum values) 140% to 120% 7% 7% 

>140% 5% 5% 120% to equal 0 20% 

140% to 120% 5% 14% equal to 80% 33% 40% 

120% to equal 5% 19% 80% to 60% 47% 20% 

equal to 80% 52% 43% <60% 7 % 13% 

80% to 60% 33% 14% 

<60% 0% 5% 

Table 5. Distribution of maximum 


values obtained in flight relative to 
Table 4. Distribution of maximum simulation for elevated-g <|>- 
values obtained in flight relative to captures 
simulation for 1-g ^-captures. 
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Figure 7 Multi-axis gross acqul 
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0 20 40 , 60 80 100 18 20 22 24 26 28 

Tim*, mc Tim*, tac 

Figure 8.- Longitudinal control, maneuver 6.2, pilot A. Figure 9.- Longitudinal control during capture, 

maneuver 6.2, pilot A. 
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Tim*, hc 

Figure 14.- Tracking reversal, maneuver 5.5, pilot B. 



Tim*. IK 

Figure 15.- Tracking results in simulator, maneuver 5.5, pilot B. 
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Figure 18b.* Elevated-g 4) captures, maneuver 8.3, pilot A. 


- .■■■■ flight reiutta equal to atm. ruufta 

flight raauKs aqua I W-20% of aim. raaufls 

Wight reautta aquat W-40% of atm, raaulta 



p (simulator), dag/sec 



Figure 19.- Summary of maximum roll rate and roll rate acceleration for 
elevated-g t> captures during roll back and roll out sub-phases. 



1049’ 

840 foot lateral offset 

r 

extended runway centerline j 

i 

offset locall 2 er/glideslope approach path 

X 

"* T “ “ 1 

30 deg. localizer 

— o — -o 

j/ / 

' simulated k>callzer 

intercept angle / 

transmitter 


simulated gfldestope 
transmitter 


K 


localizer/glideslope 

localizer/glideslope 

intercept 

tracking 


sidestep to landing 


appr oximatefy 1 .7 nm 


X=gtidestope Intercept at 2000 teet AGL at 6nm from runway threshold 

Figure 20.- Definition of ILS-offset approach and maneuver sub-phases. 
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Diagram of ILS HUD guidance showing Diagram of ILS HUD guidance 

important guidance elements with indicating no-error condition, 

hypothetical errors. 


Figure 21.- Diagram of ILS HUD guidance employed for ILS maneuver. 





DuUre* from thruhold (ft.) 

Figure 23.- Sidestep to landing, longitudinal performance, pilot A. 



Distance from threshold (ft.) 

Figure 22.- Sidestep to landing, lateral performance, pilot A. 
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